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ABSTRACT 

Most systems of administrative control include both 
formal and informal, explicit and implicit evaluative mechanisms that 
sense deviation from standards and activate corrective actions to 
return subordinates to acceptable levels of performance or to correct 
beliefs.c A^^^ showed, however, that few school 

districts have specif ic: standards against which they assess principal 
perfprmahce and outputs. This paper describes a study aimed at 
correcting some of these deficiencies (1) by employing the Natriello 
and Dor nbusch conceptual model of evaluation , which identifies 
clearly delineated evaluation stages of allocating tasks, setting 
criteria, sampling performance and/or outputs, and appraising; (2) by 
exarainirig all stages in the evaluation process, and (3) by gathering 
information from both superintendents and principals, A questionnaire 
sent to a random sample of 200 principals and to all 142 
superintendents in a southern state yielded 149 usable returns from 
principals (74%) and 121 usable returns from superintendents (85%) , 
Results showed that (1) the evaluation system is perceived to be 
specific and clear in task allocations; (2) a level of uncertainty 
exists regarding criteria setting; and (3) principals and 
superintendents disagree on how performance and outputs are sampled. 
More adegviate communication between superintendents and principals is 
needed. Appended are two pages of references, two tables, and one 
figure, (IW) 
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Pitfiills in the Evaluation of Principals 
Introduction 

' Managers are key actors in organizations providing the linkage between 
the technical core and the institutional level of the organization. They 
coordinate the flow of resources, structure productive elements, and foster 
actions which promote organizational goal attainment (Parsons, I96O; 
Thompson, 1967). In educational organizations, principals are middle 
managers who must coordinate and direct what Thompson (I967) calls "streams 
of institutionalized actions." Educational managers face a wide array of 
constraints and. contingencies which shape their behavior and influence their 
normative development (Dwyer, I985). Some of the most important forces on 
principals involve the actions taken by superior3 to control and direct 
principals and their schools. These forces emanate from various controls in 
the administrative control and evaluation systems of school 
districts(Crowson and Morris, I985; Peterson, 1984; Murphy, Hallinger, & 
Peterson, I985). 

Superiors institute systems of control to shape the norms, values, and 
behaviors of subordinates and to restrict the flow of resources to 
individuals and organizational units (Peterson, 1984). These systems of 
controls have formal and informal deviation-sensing evaluative mechanisms 
embedded in their structures. These evaluative mechanisms sense whether 
subordinates are following rules carefully, using resources effectively, 
behaving and acting in organizationally approved ways, conforming to the 
norms and values of the district, and satisfying powerful reference groups 
in the environment (Peterson, 1984). 
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Evaluative mechanisms sense deviation from a set of explicit or 
implicit standards and activate corrective actions to return subordinates to 
acceptable levels of performance or to correct beliefs. In many ways these 
mechanisms act like the thermostat in a house which senses the temperature 
in various rooms and activates either the heating or cooling system (Katz & 
Kahn, 1966). In organizations, these standards are in areas of performance, 
output, norms, and beliefs. 

The evaluative mechanism ;f in school districts face a much more complex 
set of problems than the simple house thermostat* In school districts the 
"thermostat" must assess a complex set of conditions^ try to determine what 
is the acceptable "standard" and then attempt to activate resources to 
correct the deviation, resources which are often not available or difficult 
to activate. Nonetheless, these mechanisms are in place and work in 
various ways to shape the work of principals by signaling expectations and 
by reshaping behaviors and output following deviations from the standard. 

Systems of administrative control include both formal and informal, 
explicit and implicit evaluative mechanisms. While the informal and implicit 
evaluative mechanisms shape the work of principals, in this paper we will 
examine only formal, explicit mechanisms found in the evaluation systems 
used to direct and assess principals. 

In this paper we will describe the components of a state-mandated 
evaluation system for principals using self reports from principals and 
superintendents regarding how those components are implemented. While many 
have examined the evaluation of teachers (Dornbusch & Scott, 1975; Natriello 
& Dornbusch, I98I; Darling-Hammond, Wise, & Pease, I983), few have 
systematically studied the evaluation of principals using a theoretically- 
based conceptual framework (Duke & Stiggins, 1985). 




Conceptual Fraaework , 

One of the most highly developed and empirically tested conceptual 
frameworks is found in the work of Dornbusch and Scott (1975) and Natriello 
and, Dornbusch (1981). Their model of the evaluation process posits a set of 
clearly delineated stages. These stages include allocating tasks, criteria 
setting, sampling performance and/or outputs, and appraising (Natriello & 
Dornbusch, 1981, p. 1) 

Insert Figure 1 about here 

Each atage represents a different group of specific tasks which guide and 
shape the effectiveness, reliability, and acceptance of evaluations by 
subordinates (Natriello & Dornbusch, I98I). 

In the "allocating of tasks" stage superiors assign a set of tasks to 
principals. When tasks are "active", that is when the "resistance to 
successful performance is highly unpredictable" '(e.g., remediating an 
incompetent teacher) then allocation occurs most often through delegation 
(Natriello & Dornbusch, I98I). When superiors delegate responsibility for a 
task, the subordinate is granted autonomy to decide the means to achieve the 
ends but may be held accountable for results. When tasks are "inert", that 
is when "resistance to successful performance is highly predictable" (p.2) 
(e.g., preparing attendance reports) then allocation occurs through 
directive or the specification of procedures. Directives and what Peterson 
(1984) calls behavior controls, specify what principals are to do, under 
what conditions, and at what time. These offer little discretion. 

In the "criteria setting" stage superiors, sometimes with consultation 
of subordinates, establish the standards (criteria) on which the subordinate 
will be Judged. Subordinates need to know both what they will be Judged on 
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and the level they should achieve in the task. It follows that it is 
difficult to specify criteria for some tasks (e.g., hiring effective 
teachers) and relatively easy to set criteria for other tasks (e.g., 
budgeting enough for classroom materials). Ouchi (1979) notes the 
difficulty of controlling through the evaluation of outputs when the 
criteria for Judging outputs are difficult to measure. If principals, like 
teachers, are unawars of the criteria or the standards they are to reach, 
they feel more vulnerable to the caprice of superiors and, alternately, may 
not direct their energies in the directions desired by the organization 
(Natriello & Dornbusch, 1981; Peterson, 1984). 

During the "sampling performance and/or outputs" stage superiors 
monitor performance or outputs to gather data on task accomplishment and the 
production of goods and services. The frequency of this monitoring and the 
number of sources used to sample behavior and output affect the degree to 
which subordinates view the evaluation as soundly based and assess the 
reliability of the appraisal stage (Natriello & Dornbusch, 1981). 
Performance monitoring is often used to sample inert tasks and output 
monitoring for active tasks. 

In the "appraisal" stage superiors eissess the degree to which 
subordinates have achieved the standards or reached the criteria established 
for performance or output which was specified in an earlier stage (Natriello 
& Dornbusch, 1981). In this stage the clarity, specificity and reliability 
of the prior stages affect the precision and efficacy of the appraisal. If 
task allocation was not clear, then principals may not have worked on the 
central tasks. If the criteria for appraisal were not specific, then 
principals may have worked toward criteria they selected (See Turcotte, 1974 
for a discussion of this phenomena). If performance or output samples were 



; ; biased . or unreliable, then the appraisal will not be based on a valid sample 
■ of performance or outcomes. Thus, even a highly developed appraisal 
mechauiism may produce an inaccurate or incorrect evaluation of performance 
or output.\Yet without an appraisal stage, the evaluation process is 
incomplete. 

The clarity and precision of each of these evaluation stages influences 
the quality and effectiveness of the evaluation. We need to understand how 
each part is enacted to understand the whole. Organizational control through 
evaluation is comprised of several parts, each of which contributes to the 
whole. The parts must interact to produce reliable and effective 
evaluations or unreliable, counterproductive assessments. Present 
conceptually driven, empirical knowledge of the evaluation process for 
principals is incomplete (Duke & Staggiris, 1985? Harrison, I985). Prior 
studies have noted that the tasks of principals are diverse, complex, and 
often specified in general Job descriptions. Principals frequently report 
they are not sure what they are supposed to be doing (Peterson, 1982, 1984). 
With this type of work, the allocation process is often unclear. 

Descriptive studies suggest that principals often are unsure what 
criteria superiors use to evaluate their actions and outputs or which 
criteria carry the most weight (Duke & Stiggins, 1985; Peterson, 1984). 
This suggests that the criteria setting stage is frequently neither made 
explicit nor clear. 

Existing research also finds that sampling of performance and outputs in 
many districts is infrequent, unsystematic, and relies more on opinions of 
important reference groups than on quantifiable measures of performance or 
output (Duke & Stiggins, I985; McDowell, 1954; Peterson, 1984). The 
sampling stage thus remains problematic for many principals. 



Finally, the few studies which identify actions by superiors in the 
appraisal stage note that while some districts employ a systematic approach 
to assessing student performance, accomplishment of objectives in a 
management-by-objectives system, or the quality of teacher evaluation, a 
largfe proportion of districts do. not have specific standards against which 
they assess principal performance and outputs (Duke & Staggins, 1985; 
Peterson, 1984). Often the appraisal 3feai-^ as in other organizations 
depends on reference group assessments, intuitive evaluations of 
performance, and diffuse criteria of effectiveness (Thompson, I967). 

Prior studies provide an incomplete picture of the evaluation process. 
None use a conceptual framework to guide the investigation. None look at 
the four stages of the evaluation process. And some do not query 
subordinates about the system. In this study, we will try to correct some 
of these deficiencies, building on these useful prior studies, by employing 
the Natriello & Dornbusch (I98I) conceptual model, by examining all stages 
in the evaluation process, and by gathering information from both 
superintendents and principals. 
Methodology 

In order to increase the generalizability of results, we sampled a 
large group of superintendents and principals. Given the wide range of 
practices across school systems, we selected a state where a standard 
principal evaluation process had been adopted. This evaluation system 
provided general outlines of the evaluation process, but left some 
discretion in the hands of local districts. By having a large sample we 
increased our generalizability, but lost the rich detail of case studies 
(Yin, 1984). 



Nonetheless, we must note some of the potential limits of self-report 
data as suggested by an anonymous reviewer to the AERA proposal. First, 
while principals have an interest in reporting accurately about the ways 
they are evaluated, superintendents may inaccurately report what they do 
sincfe they are more vulnerable to sanctions if the study discovered 
substantial digressions from stated policies (Anonymous Reviewer), Attempts 
were made to ameliorate this problem by placing no identification numbers on 
the survey. Nonetheless, superintendents' self-reports may be biased in an 
administratively positive direction. Second, there are always problems in 
drawing inferences and comparing groups on Likert scaled responses (for 
example, principals and superintendents may have different perceptions of 
what "frequent supervisory visits" means) and self-reported rankings of the 
relative importance of various criteria (e.g., the relative importance of 
various "results" of principals' work). These problems do limit the degree 
to which we can describe the actual administrative behaviors and the actual 
application of criteria when principals are evaluated. Nonetheless, they 
provide a mapping of beliefs about and perceptions of this process, a 
mapping which can guide further research on the process as it is occurs in 
districts. 

Population and Sample 

Questionnaires were sent to a simple random sample of 200 principals in 
a southern state and to all 142 superintendents. Of those sent 
^questionnaires, 74 percent of the principals (N= 149) and 85 percent of the 
superintendents (N=121) returned them in usable condition. 
Instrument and Data Analysis 

Based on the methodological approaches taken by earlier studies, a. 
questionnaire was designed to gather information about the ways 
superintendents evaluate principals. The questionnaire used open-ended and 



Likert scaled questions to gather data on criteria used in the evaluation 
process,- the focus and purpose of evaluation, the sources of information 
used, and the results or outputs principals perceive to be important to 
superintendents. ' 

Responses to open-ended questions were categorized and frequencies 
tallied, while means for scaled questions were calculated (with "strongly 
disagree" = -2 to "strongly agree" = 2). Descriptive statistics were used 
to show the patterns of the response of principals and superintendents 
separately. A comparison of percentage differences showed discrepancies 
between principals* and superintendents' responses oh some items. 

As we mentioned above, we did not place identification numbers on the 
questionnaires in an effort to increase response x'ate and reliability of 
superintendents' responses^This decision made it impossible to compare the 
responses of principals to those of their superintendents. Rather, 
comparisons will be between superintendents and principals as groups. 
Further studies should attempt to- match principal and' superintendent 
responses by district. 

This paper discusses the major stages in the evaluation process. We 
begin by discussing the allocation of tasks and the setting of criteria, 
then turn to the sampling of performance and outputs, appraisals and the 
ways superintendents communicate them to principals. Finally, we will 
discuss the implications of this research for both theories of the 
evaluation process and for the practice of evaluation in educational 
enterprises. 

Allocation of Tasks and Setting of Criteria 

In their model of the evaluation process, Natriello and Dornbusch 
(1981) identify the allocating of tasks as the first component of the 



procedure, followed by the setting of criteria for performance. 
Descriptions of the tasks of the principals were set forth by the state 
board of education when it adopted a Job description of the principal in 
1981. The Job description stated the purpose of the principal as follows: 
• To serve as the chief administrator of a school in developing and 
implementing policies, programs, curriculum, activities, and budgets in 
a manner that promotes the educational development of each student and 
the professional development of each staff member. In addition the 
board identified 23 duties and responsibilities of the principal. 
Furthermore, the state performance appraisal instrument outlined five 
major task domains of the principalship. The domains include (a) General 
Planning and Oversight; (b) School and Classroom Objectives; (c) Personnel 
Organization and Management; (d) Clientele Relationship and Their 
Management; and (e) Allocation of Supplies, Equipment, and Support Services. 
Each of these domains contain from two to four major functions, with several 
sub-functions specified for appraisal purposes. 

Both the superintendents and principals responding to the survey 
believe that the major functions, outlined above, provide an accurate 
description of the principal's role. Nearly 81 percent of the 
superintendents and 72 percent of the principals responding concurred. 
However, they are not as certain that the appraisal instrument makes the 
criteria for principal performance clear. A slight majority of the 
superintendents (51?) believe that criteria for performance is clear. 
Overall they report some uncertainty. 

Natriello and Dornbusch (I98I) contend that criteria for performance 
appraisal must be specific enough to allow for consistency among 
evaluators. One item on the questionnaire dealt with the superintendents' 
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interpretation of criteria for principal performance. When asked if 
superintendents from system to system in the state agree upon the criteria 
for principal effectiveness, superintendents and principals reported a 
higher level of uncertainty than any other item in the questionnaire. The 
perception is that superintendents from system to system do not agree upon 
the criteria for principal effectiveness. 

In order to perform successfully, workers must have a clear 
understanding of their superiors' expectations. Nearly 80 percent of the 
superintendents reported that they make their expectations of principal 
performance clears While the majority of principals concur (5856), nearly 42 
percent are uncertain or feel that expectations are not made clear to them. 
Superintendents are not communicating their expectations to principals as 
successfully as . they believe they are. 

Along similar lines, the two groups differ in their opinions regarding 
the results or outcomes of the principalship considered to be important to 
the superintendent. Earlier research (Peterson, 198^1) identified results of 
the principalship that principals believe are important to central office. 
Using a similar list, we asked respondents to rank eight different results 
in order of perceived importance to their superiors. The two groups agreed 
on which were the top five, but differed considerably on the order of the 
five (Table 1). 



Insert Table 1 about here 

Principals perceive the reaction of the public, positive or negative, 
to be the most important indicator of principal performance to the 
superintendent. This agrees with Peterson's (198^1) findings. 
Superintendents, on the other hand, agree that public reaction is important. 
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but they rank it fifth. Eighty-one percent of the principals mentioned 
public reaction as one of the top five results to only 58 percent of the 
superintendents, suggesting greater saliency in the perceptions and lives of 
principals., 

''Judging from the top three items ranked by principals-- public reaction, 
atmosphere of the school, and teacher performance and morale, it appears 
that principals believe that superintendents are concerned with how their 
work is perceived by others* Principals rank "quality of instruction," 
criteria which is difficult to measure, fourth. While superintendents 
ranked it first. Student performance as measured by test results is ranked 
fifth by principals and fourth by superintendents, far down on the rankings. 
Clearly, instructional performance is perceived as more important than 
instructional outcomes. 

There was more agreement among the superintendent group than the 
principal group regarding the results of principal performance watched by 
the superintendent. As we see in Table 1, while the number one result 
identified by the principals was mentioned by ffl percent of the group, 87 
percent of the superintendents mentioned their number one response (general 
quality of instruction), Furthermor-Cj while 79 percent of the principals 
identified the atmosphere of the school as being among the top five results 
watched by the superintendent (giving it the number two ranking among 
principals), a higher percentage of superintendents (82?) included this 
result in their list even though it was ranked only number three amongst 
their group. 

To further emphasize the point, a larger proportion of superintendents 
checked three results (general quality of instruction, teacher performance 
and morale, and atmosphere of the school) than were checked by principals' 
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for their response ranked number one. It may be that while superintendents 
have a clear idea of what results or outputs they desire of principals^ 
communication of these expectations are not being received by principals. 
In shorty, vhile these tasks were allocated, criteria were not understood 
by ppincipalSo 

Major differences between the two groups appear in their perceptions of 
the importance of four results. The principals perceive that public 
reaction is the most important indicator of their performance for the 
superintendent. Eighty-one percent of the principals cited these reactions 
to only 58 percent of th2 superintendents. 

Other differences point to disagreement between these two groujps about 
criteria. The largest percentage difference in responses deals with the 
principals "not making waves." While both groups ranked this result eighth, 
only 3 percent of the superintendents mentioned it as compared to 29 percent 
of the principals- — a 26 percent difference. The fourth largest difference 
(21 percent) is the criteria "general quality of instruction." The 
superintendents ranked this number one with 87 percent of the group 
mentioning it, while in the principals top five results, only 66 percent 
mentioned it. Here we see that principals perceive different criteria than 
do their superiors. These differences may affect their work. ^ 

Subordinates tend to focus their energies • apply resources, and solve 
problems they perceive to be important to their superiors, and which are t.he 
focus. of evaluation (Turcotte, 1974). This makes it important that 
performance expectations and criteria used in the evaluation process are 
understood by subordinates. Beliefs about what is important will influence 
the work of administrators, whether or not their beliefs are accurate. Our 
findings reveal that both superintendents and principals are somewhat 
uncertain regarding the criteria for principal performance as set forth by 
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the principal performance Appraisal instrument. Furthermore, principals and 
superintendents have substantially different beliefs 'about what criteria are 
applied and in what order of importance. In addition, as Natriello and 
Dornbu3ch (1981) point out regarding teacher evaluation, evaluators often 
assume that those they are evaluating understand what is expected of them 
and know the performance or outputs on which they are being evaluated. Our 
data reveal that while superintendents believe they are communicating their 
expectations to principals, 42 percent of the principals are uncertain or do 
not understand what their superintendents expect of them. 

For effective evaluation to occur, superintendents must communicate 
their expectations of performance to principals in ways these expectations 
can be received accurately. Otherwise, principals are likely to direct 
their efforts in the areas they only partially understand or intuit to be 
important to the superintendent (i.e., the principal spending more time as a 
public relations person than an instructional leader or applying energies 
toward appeasing their reference groups^-- -teachers and PTO~demands, or 
toward accomplishing personal, self-aggrandizing ends) (Downs, 1967). 
Sampling of Performance or Outputs 

The sampling of performance or outputs stage follows the allocation of 
tasks and setting of criteria in the Natriello and Dornbusch Model. The way 
information is collected to assess performance is an important component of 
any assessment system. Equally important is that the evaluatee understands 
the data collection process. 

Only 51 percent of the principals claim to know precisely how 
superintendents gather information used in their evaluations. The 
administrators were provided with a list of eight methods superintendents 
use to gather data for principal evaluations (Peterson, 1984). Both groups 
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were asked to rank the top five methods they thought superintendents used to 
gather data on principal performance and outputs (Table 2). These data 
specify the perceived sampling procedures of superiors. 

Insert Table 2 about here 

Few principals report knowing precisely how the superintendent gathers 
data to evaluate their performance. Nonetheless, when asked to check which 
sources they thought were used, 87 percent said they thought superintendents 
used community and parents as one source of information. It ranked in the 
top five sources reported. This is consistent with their ranking of public 
reaction as the number one indicator of principal performance watched by the 
superintendent. Superintendents also mentioned the community and parents as 
being an important sourc<^ ^^f information, but not ' to the extent reported by 
principals. 

As Table 2 indicates, the groups rank the major sources superintendents 
utilize in the appraisal process about the same. They generally agree on 
how performance and outputs are sampled. Major differences lie in the 
relative frequency that community and parents and board members are 
mentioned as sources of information. Fifty percent of the principals 
mentioned board members as one of the top five sources of information, while 
only 31 percent of the superintendents did so. 

Another key dimension of the sampling stage is frequency of sampling. 
The most direct method of sampling principal performance is by visiting the 
school and observing the principal at work. Superintendent and principal 
responses reveal a major difference of opinion concerning the frequency of 
direct supervision. 
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As one principal writes, "It is extremely important that the 
superintendent make several visits to the school. Unfortunately, ours does 
not." Another wrote, "Last year I was never visited by the evaluator, and I 
received all 'superior performance* ratings — not effective or helpful to 
me." These principals may not view the evaluations as soundly based because 
sampling by .observation was infrequent. Nearly 81 percent of the superintendents 
report that they frequently visit the school, while only 37 percent of the 
principals report this. The majority of principals {60%) reported 
infrequent visits compared to 17 percent of the superintendents— a 43 
percent difference. In general, subordinates often view evaluations as 
unsoundly based when their work is infrequently sampled (Natriello & 
Dornbusch, I98I). 

Although the particular appraisal instrument we examined is generally 
more detailed and specific than most, some inconsistencies in implementation 
are occuring, in part perhaps, due to the special nature of principal's work 
(Peterson, 1978). Findings reveal that while superintendents think they are 
promoting instructional leadership, principals may be receiving the message 
that relations with the community and school board are more important. 
Similarly, while superintendents perceive that they frequently visit the 
school, principals perceive less frequent visiting. These differences 
suggest that there may be substantial discrepancies in the ways superiors 
and subordinates view performance sampling. 
Appraisal and Communications to Principals 

Communication, or its lack, is a recurring theme in this study. The 
communication of expectations was problematic during the allocation of tasks 
and the setting of criteria stages. Only 58 percent of the principals 



reported 



that the communication of superintendeots* performance expectations 



was clear, contrasted with nearly 80 percent of the superintendents who 
believed this. Similarly, we found a discrepancy regarding indicators of 
principal performance and their importance- to the superintendents (Table 1). 

The communication of expectations is a critical component of an 
evaluation system, if principals believe that the superintendent is more 
concerned with public reaction and building management than instruction, 
they will pattern their behavior to conform to their perceptions of the 
superintendent's expectations. 

Frequent communication to subordinate is key during every stage of the 
evaluation process. In communicating results of appraisals, both positive 
and negative feedback are required to improve performance. Both groups in 
this study report that superintendents frequently communicate satisfaction 
with principal performance. While the two groups generally agree that the 
superintendent frequently communicates satisfaction, there is disagreement 
regarding the frequency of communicating dissatisfaction. 

The groups disagreed on how frequently superintendents communicated 
dissatisfaction with principal performance. Superintendents reported they 
frequently communicated dissatisfaction with principal performance, while 
principals reported that dissatisfaction was not frequently communicated. 
Eighty-three percent of the superintendents perceived that they frequently 
communicated satisfaction, while only 59 percent reported themselves as 
frequently communicating dissatisfaction. This may be the case for several 
reasons. There may be a reluctance on the part of the superintendent to 
express dissatisfaction and so they do not communicate it; they may believe 
they are giving negative feedback more frequently than they are; or 
principals may be interpreting as neutral or positive some of the negative 
communications sent them by superintendents. Whatever the reason. 
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principals do not believe they are receiving negative feedback as frequently 

as superintendents believe they are communicating such feedback. 

Summary 

The data provide a comprehensive view of the evaluation system system 
of jfrincipals when it is formal, standardized, and complex. In summary, we 
find: 

1. Superintendents have a more consistent perception than do the 
principals of the process and generally feel more positive toward it. 

2. Principals report being less clear regarding that what is actually 
occurring in the evaluative process as indicated by a wider distribution of 
principal responses. 

3. Principals and superintendents do not perceive the same actions 
occurring in the evaluation of principal performance. 

4. Principals believe that superintendents rely heavily upon 
community opinions as a gauge of principal effectiveness, while 
superintendents report less reliance on these opinions. 

5. Superintendents report employing a number of methods to gather 
data for the evaluation of principals. More principals report that the 
community is the major source of data, while superintendents report that the 
principal is their major source of information. 

6. Superintendents report that they are are most concerned with the 
quality of instruction, while principals feel that public reaction and 
management are the major concerns of the superintendent during evaluations. 

The findings indicate that (a) the evaluation system is perceived to be 
clear and spc^X^ic in the allocation of tasks, (b) a level of uncertainty 
exists rega:^ifll-ng the setting of criteria, and (c) principals and 
superinten(5^*iv3 disagree on how performance and outputs are sampled. 
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Similar to what Peterson (I98M) found, principals perceive parents to 
be a key source of evaluative information. They report that management and 
public reaction are two of the most important "results" of principals* work 
(Tables 1 and 2). In contrast, superintendents report other sources of 
information (e.g., the principal) as being central. Most superintendents 
(8356) believe that instructional leadership is more important than 
management or public relations while 43 percent of the principals believe 
their superiors feel that management is more important. 

Furthermore, both principals and superintendents agree that 
satisfaction with performance is frequently communicated. In contrast, 
principals report that superintendents do not communicate dissatisfaction as 
frequently as superintendents believe they do. 

Overall, even with structured, state-mandated principal evaluation 
systems we find pitfalls in the evaluation of process Just as Peterson 
(I98M) and Duke and his associates (I985) found in less structured systems. 
Criteria are often not communicated to principals clearly, sampling is 
sometimes infrequent or dependent on biased sampling procedures or 
potentially unreliable providers of data, and appraisal uses standards 
derived more from reference group assessments than quantitative appraisals 
of behavior and performance. 

Discrepancies in educational evaluation systems seem to occur during 
several stages: during the setting of criteria, the sampling of performance 
and outputs and when feedback is communicated. These are key stages of the 
evaluation process. If principals are to improve their performance, these 
stages should be clear, specific, and known by both evaluator and evaluatee. 

The study reveals that many of the pitfalls of evaluating principals 
(Natriello & Dornbusch, I98I) also are found in the evaluation of teachers. 
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To overcome these pitfalls superintendents must make their expectations of 
principal performance clear, insure that principals understand the tasks 
they are to accomplish, the criteria used to assess performance, the type of 
data used, and the ways performance and outputs are appraised. 
Additionally, more frequent communication regarding principal performance, 
with superiors communicating dissatisfaction as well as satisfaction, would 
help improve the work of these managers by providing useful information on 
problems to correct, outputs to maintain, and tasks to improve. If these 
organizational thermostats are to work to improve the effectiveness of 
principals, then their inner workings must be fine-tuned and calibrated for 
the complex world of schools and school management. 

Considerable further research should take off from this study to 
further our understanding of the evaluation process. First, a study should 
collect data and compare responses of superintendents to that of their 
principals, looking for organizational and behavioral characteristics which 
distinguish dyads with consistent perceptionr and those with discrepant 
ones. District size as well as the communication medium of the 
superintendent may affect the degree to which subordinates receive and 
decode accurately messages about Job performance. Second, research should 
be conducted which examines on site the evaluation process of principals. 
While a costly investigation, it would provide an accurate picture of the 
behavioral realities of the process. In addition, the differential affect of 
evaluation on principal performance and outcomes could be collected and 
compared to the ways different stages of the process are enacted. Finally, a 
study should be conducted which examines the relationship between perceived 
criteria used to assess principals, principal time use, and student academic 
achievement. It seems reasonable to assume that, on average, principals will 
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apply their energies and expend their time on activities they believe are 
important to superiors and which they believe are evaluated. Where the 
criteria are perceived to be student per f ordnance, we would expect principals 
to spend more time on tasks related to instruction, with student achievement 
incr^'easing over time. Alternately, where the criteria are perceived to be 
administrative work or public reaction, we would expect principals to spend 
their energy and time on increasing their scores on these results, with 
instruction and student achievement faltering. 

Teacher evaluation, has gained considerable attention in recent years. 
It is time to apply more attention to the question of how superintendents 
evaluate principals, under what conditions, and to what end. With more 
knowledge about this process, practitioners will be able to improve the 
assessment and direction of some 100,000 school principals and scholars will 
be able to increase our understanding of the constraints on managers in 
educational organizations. 
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Table 1 



Result/,^ of the Principalship the Superintendent Views 
aa Indicators of Principal Performance 



Superintendents Principals 



Results 


Rank 


M 


%li 


Rank 


M 


HM 


General quality of 
instruction 


1 


99 


87 


4 


96 


66 


Teacher performance 
and morale 


2 


96 


84 


3 


102 


70 


Atmosphere of 
school 


3 


94 


82 


2 


114 


79 


Student performance 
and progress: test 
scores 


i| 


79 


69 


R 


Ql 
7 1 




Public reaction: 

positive or negative 


5 


66 


58 


1 


118 


81 


Student behavior and 
principal's relations 
with students 


6 


63 


55 


7 


62 


43 


Adherence to system 
rules and procedures 


7 


30 


26 


6 


69 


48 


Not "making waves" 


8 


4 


3 


8 


42 


29 


Others 


8 


4 


3 


9 


5 


3 



Rank = indicates the ranking of the result by respondent group 

M = number of times a response was mentioned, subjects were given 
list with all items and asked to rank the top five 

^ M = thn percentage of respondents mentioning a particular result 
(These data are reported in Harrison, 19850 



Table 2 



Perceived Sources of Data Superintendents Use 
for the Evaluation of Principals 



Superintendents Principals 



Information Source 


Rank 


M 


5JM 


Rank 


M 


StM 


The principal directly 


1 


106 


87 


2 


112 


75 


Central office 
personnel 


2 


92 


76 


3 


101 


68 


Superintendent 


3 


83 


69 


5 


85 


57 


Community and parents 


4 


82 


68 


1 


129 


87 


Teachers 


5 


78 


64 


It 


94 


63 


Reports, written 
materials 


6 


67 


55 


6 


75 


50 


School Board 


7 


38 


31 


6 


75 


50 


Test scores 


8 


15 


12 


6. 


26 


17 


Others 


9 


6 


5 


9 


a 


6 



Rank = indicates the ranking ^/ the information by respondent group 

Ms number of times a response was mentioned, subjects were given a 
list with eight information sources and asked to rank the top five used by 
the superintendent 

%li = the percentage of respondents mentioning a particular item 



(These data are reported in Harrison, 1985.) 



